Many video systems employ image processing capabilities, such as scaling, deinterlacing, and the like, for video processed by the video system. Typically, a video processing system encodes, decodes, or transcodes the video data and stores the resulting video data in system memory. The video data is then subsequently accessed from the system memory for image processing. This approach thus consumes considerable memory bandwidth as the video data is pulled from the system memory for processing, as well as introduces considerable latency in completing the image processing.